1 Simulation II Main Paper

Placeholder

1.1 Value Function

1.2 Matern 5/2 (n=500)

1.2.1 Estimated Value Function

1.2.2 Interpolating

1.2.3 Homoskedastic

1.2.4 Heteroskedastic

2 Simulation III Main Paper

Placeholder

2.1 Value Function

2.2 Matern 5/2 (n=500)

2.2.1 Estimated Value Function

2.2.2 Interpolating

2.2.3 Homoskedastic

2.2.4 Heteroskedastic

3 Simulation II Appendix

Placeholder

3.1 Matern 5/2 (n=1000)

3.1.1 Estimated Value Function

3.1.2 Interpolating

3.1.3 Homoskedastic

3.1.4 Heteroskedastic

3.2 Matern 5/2 Heteroskedastic (n=1000)

3.2.1 Estimated Value Function

3.2.2 Interpolating Model

3.2.3 Homoskedastic Model

3.2.4 Heteroskedastic Model

3.2.5 Residuals

3.3 Matern 3/2 (n=1000)

3.3.1 Interpolating

3.3.2 Homoskedastic

3.3.3 Heteroskedastic

3.4 Matern 5/2 (n=1000) Log-Normal

3.4.1 Interpolating

3.4.2 Homoskedastic

3.4.3 Heteroskedastic

4 Simulation III Appendix

Placeholder

4.1 Estimated Value Function

4.2 Matern 5/2 (n=1000)

4.2.1 Interpolating

4.2.2 Homoskedastic

4.2.3 Heteroskedastic

4.3 Matern 3/2 (n=1000)

4.3.1 Interpolating

4.3.2 Homoskedastic

4.3.3 Heteroskedastic

4.4 Matern 5/2; Log-Normal Prior (n=1000)

4.4.1 Interpolating

4.4.2 Homoskedastic

4.4.3 Heteroskedastic

5 Case Study

5.1 Value Surface

5.1.1 Estimated Value Function Using Normalized IPW

(#fig:unnamed-chunk-3)Estimated value of regime using normalized IPW

5.1.2 value Surface using MSM

(#fig:unnamed-chunk-4)Estimated value surface of regime using quadratic MSM and scatter plot of normalized IPW estimates.

5.2 Matern 5/2 GP Graphics

5.2.1 Interpolating GP

(#fig:unnamed-chunk-5)Estimated value of regime using interpolating GP after an additional 25 sampled points

5.2.2 Homoskedastic GP

(#fig:unnamed-chunk-6)Estimated value of regime using homoskedastic GP after an additional 25 sampled points

5.2.3 Heteroskedstic GP

(#fig:unnamed-chunk-7)Estimated value of regime using heteroskedastic GP after an additional 25 sampled points

5.3 Matern 3/2 GP Graphics

5.3.1 Interpolating GP

(#fig:unnamed-chunk-8)Estimated value of regime using interpolating GP after an additional 25 sampled points

5.3.2 Homoskedastic GP

(#fig:unnamed-chunk-9)Estimated value of regime using homoskedastic GP after an additional 25 sampled points

5.3.3 Heteroskedastic GP

(#fig:unnamed-chunk-10)Estimated value of regime using homoskedastic GP after an additional 25 sampled points

5.4 Matern 5/2 Covariance; Log Normal Prior

5.4.1 Interpolating GP

(#fig:unnamed-chunk-11)Estimated value of regime using interpolating GP after an additional 25 sampled points

(#fig:unnamed-chunk-11)Estimated value of regime using interpolating GP after an additional 25 sampled points

5.4.2 Homoskedastic GP

(#fig:unnamed-chunk-12)Estimated value of regime using homoskedastic GP after an additional 25 sampled points

(#fig:unnamed-chunk-12)Estimated value of regime using homoskedastic GP after an additional 25 sampled points

5.4.3 Heteroskedastic GP

(#fig:unnamed-chunk-13)Estimated value of regime using homoskedastic GP after an additional 25 sampled points

(#fig:unnamed-chunk-13)Estimated value of regime using homoskedastic GP after an additional 25 sampled points